METHOD AND SYSTEM FOi 




The present invention relates to a method for inspecting the surfaces of a three- 
dimensional body, with which at least one camera and at least one illuminating device 
5 are moved relative to the surface of the object, pictures are taken of the areas to be 
inspected on the surface during the movement of the camera relative to the surface, 
and the pictures are transmitted to a computer and evaluated therein. The invention 
also relates to a corresponding system for inspecting surfaces. 

The three-dimensional object can be a body or a body part, in particular. In this case, 
10 the surface to be inspected is often a painted surface or a bare sheet-metal surface, the 
quality of which is to be inspected. The inspection is carried out to identify defects or 
flaws, such as topological defects, on the surface using optical scanning. The camera 
used for this purpose can be any optical picture-taking device that meets the particular 
requirements and is optimally adapted to the surface to be investigated. 

15 A similar inspection system is known, e.g., from US 5,142,648. with which a large 
number of illumination units and cameras is installed along a production line for 
passenger vehicles, the signals of which are evaluated in an arithmetic logic unit. This 
system has the problem, however, that the entire three-dimensional object cannot be 
examined with equal inspection quality in all areas to be inspected on the surface, 

20 because the picture-taking conditions are too different for the various areas. 

The object of the present invention, therefore, is to provide a possibility for inspecting 
surfaces, with which all areas to be inspected on the surface can be examined with the 
same level of quality. 

This object is attained essentially with a method of the type described initially having the 
25 features of Claim 1 by the fact that the camera, the illumination device and the surface 
are brought into a defined geometric relationship with each other during the inspection 
of each area to be inspected on the surface, at least for the period of time required to 
take a picture. In this manner, at least one picture of each of the areas to be inspected 
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is obtained that is equal to the rest of the pictures of the other areas to be inspected in 
terms of the picture-taking situation and quality. A high level of inspection quality is 
ensured as a result. The geometric relationship to be attained can be specified for every 
inspection in accordance with the particular requirements. A control computer ensures 
5 that an inspection unit with a camera and illumination and, e.g., the surface to be 
inspected on the body, are moved relative to each other in a suitable manner. It is 
particularly advantageous when one or more inspection units and the object itself to be 
inspected are controlled in a coordinated manner, e.g., using a single control computer. 
It is then possible to perform other types of work on the object during the inspection and 

10 while the object is moving. This is particularly space-saving and is therefore particularly 
well-suited for use on complicated production or processing lines, with which the 
greatest possible number of tasks must be performed along the shortest possible path. 
The picture-taking position can be defined, in particular, by an angle at which the 
picture-taking is carried out, and/or via the resolution of the picture-taking. The 

15 resolution can be controlled via the distance at which the picture is taken, via the 
selection of the focal distance of camera lenses, or the like. 

The relative motion between the camera, the illumination and surface to be inspected 
can be produced by moving the object that is the three-dimensional body independently 
of the optical systems defined by the camera and the illumination device, by measuring 

20 this motion, and by optionally adapting a motion of the optical system to the 

independent motion of the object. This variation is advantageous particularly when the 
inspection system according to the present invention must be adapted to an existing 
production line. It is possible to also design cameras and illumination devices - which 
can be combined into a single inspection unit - to be stationary, in which case several 

25 different inspection units or cameras and illumination devices are then preferably 
provided at various positions. Furthermore, the relative motion can also be brought 
about by ensuring that the object to be inspected is stationary and then moving the 
camera with the illumination device over the three-dimensional object. A combination of 
the possibilities described above is also feasible. The relative motion can be specified 

30 particularly easily when all displacement devices are synchronized with each other. This 
makes it possible to also easily inspect the moving object, since the relative motion of 
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camera/illumination and the object is coordinated. 

It can be advantageous to combine at least one camera and at least one illumination 
device into one inspection unit and to move them together. The same illumination 
situation is attained for each camera orientation by specifying a relatively fixedly 
5 specified orientation between the camera and the illumination device. The inspection 
unit is then moved in a manner that is controlled as a function of the object motion that 
is measured. An inspection unit can then include several cameras and/or light sources. 

According to a preferred embodiment, the camera, illumination device and the surface 
are brought into a defined geometric relationship with each other during the inspection 
10 of each area to be inspected on the surface, at least for the period of time required to 
take a picture. Since certain surface defects can be found only in a certain picture- 
taking position, which can differ from the picture-taking position for another defect type, 
different defects can also be reliably detected in this manner. 

According to the present invention, the defined geometric relationship can be 
15 determined by the angle between the surface of the area to be inspected, the 

illumination and the camera and/or by the distance between the surface of the area to 
be inspected and the illumination and/or the camera. Based on the known positions of 
the three-dimensional body, the illumination and the camera, the control computer can 
identify these relationships for every area to be inspected and thereby ensure - for each 
20 area to be inspected - that the defined geometric relationship was attained at least once 
during the entire inspection. 

According to the inventive method, it is also possible to select different-sized areas to 
be inspected, depending on the curvature of the surface. This is advantageous, in 
particular, when the curvature is designed such that defects can no longer be detected 
25 unambiguously, due to the curvature. In most practical cases, however, the entire image 
can be evaluated. 

To allow easy adjustment of the geometric relationship for every area to be inspected, it 
is provided according to the present invention that the camera, the illumination device 
and/or body with the surface to be inspected are movable in one or more degrees of 
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freedom. Preferably, at least one displacement device is provided for this purpose, on 
which the camera, illumination device and/or body are mounted. Possible displacement 
devices include, e.g., a manipulator, a handling device, or a multiaxial traveling unit with 
linear traveling axes and/or axes of rotation. 

5 Depending on the structure of the surface to be inspected, the illumination can take 
place in a diffuse, directed or structured manner, as sustained picture-taking and/or a 
flash picture-taking. The illumination can be alternating dark-field and/or bright-field 
illumination and/or a - preferably directed - two-dimensional illumination. It is also 
possible to project suitable patterns. 

10 In order to also enable identification of various surface defects, it can be advantageous 
according to the present invention to take several pictures of an area to be inspected on 
the surface under various illumination situations and/or with different camera settings. 
The camera setting can include the orientation of the camera, and the picture-taking 
parameters, such as aperture, shutter speed, or the like. These parameters can be 

15 adapted, preferably automatically, to various surface properties, such as color and 
reflectivity. The same applies for the type of illumination. The surface properties are 
recognized via the image evaluation. The parameter adaptation is carried out preferably 
automatically and in a self-learning manner. 

According to a particularly preferred embodiment of the inventive method, several 
20 cameras and several illumination devices are combined to form at least two subsystems 
that are movable relative to the surface to be inspected, the subsystems being 
interconnected via a communication interface, and the inspection result being obtained 
by evaluating the images of several or all of the subsystems. A subsystem is composed 
of at least one camera and one illumination device. The subdivision into several 
25 subsystems allows a three-dimensional body to be inspected more quickly overall, since 
pictures can be taken of various areas simultaneously, and/or various geometric 
relationships can be attained simultaneously for an area to be inspected. 

An advantageous application is, e.g., the simultaneous monitoring of large and small 
surface areas. To this end, the subsystems composed of camera and illumination 
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device are each adapted to the special monitoring task. For small areas, the system can 
include, e.g., an illumination device and a camera. For a large area, several adjacent 
cameras and illumination systems can be combined into one subsystem, so that a 
partcularly large area to be inspected can be covered all at once using this subsytem. 
5 Some of the subsystems can have a displacement device for moving the subsystems, 
or they can be located such that they are stationary next to the moving object. The type 
of camera and the illumination can also differ from subsystem to subsystem. 

According to the inventive method, after the pictures are taken, they are evaluated, 
preferably with the aid of image-evaluation algorithms stored in a computer system. This 

10 image evaluation is generally known. It can be provided, according to the present 

invention, however, to except specifiable structures from detection as defects during the 
inspection. During painting, defects can occur, which are caused, e.g., by dirt particles 
located on the surface during painting or due to wetting problems on the surface. In 
accordance with the structures recognized in the inspection, the defects are then 

15 classified accordingly; this makes it possible to except certain defects from being 

recognized as defects during the inspection, or to specify various classes of defects. In 
the image evaluation, it is possible to evaluate the size, contrast, relationship between 
size and contrast, the geometry and contour, such as the identification of a surrounding 
edge of a certain contrast area, and/or the positioning of the defect in the bright field or 

20 dark field, when bright-field/dark-field illumination is carried out. The image-evaluation 
algorithms can be adapted to various surface properties, e.g., via the suitable selection 
of algorithms or parameterization. 

With the inventive method, the relative position between the surface to be inspected and 
the camera and/or the illumination device is detected, and the picture is taken such that 
25 it is controlled via resolution, position and/or time, in particular, as a function of the 
relative position. Byway of this control, which is specified, e.g., using a control 
computer, it can be ensured in a particulary simple manner that a picture of every area 
to be inspected is taken at the right time, i.e., when the relative position between body, 
camera and illumination conforms with the specified relationship. 

30 According to an advantageous embodiment of the present invention, the relative 
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position between the camera and tlie object and/or the illumination device can be 
detected by taking a picture using sensors. This picture is analyzed by the image- 
evaluation system. Based on this, the particular coordinates and positions are 
determined in an absolute and/or relative manner. Based on these automatically 
5 determined positions, the camera, object and/or illumination device can then be oriented 
automatically. For object recognition, geometric features or the appearance of the 
surface, for example, can be specified as orientation features to the image-evaluation 
system. In addition or as an alternative thereto, the positions can also be detected using 
other sensors. 

10 Finally, it is possible to inspect different specified areas to be inspected on the surface 
using different settings in the geometric relationship, the illumination situation and/or in 
image-processing parameters, when the purpose is to detect various defects, e.g., in 
the different areas. Different quality ranges can be established in this manner, for 
example. 

15 The object according to the present invention is also attained by a system for inspecting 
surfaces of a three-dimensional body, which can be used, in particular, to carry out the 
method described above. The system has a camera for taking pictures of the areas to 
be inspected on the surface, and at least one illumination device, at least one 
displacement device that moves the camera, illumination device and body relative to 

20 one another, and an evaluation unit for evaluating the pictures that were taken. A 

control device of the system is set up, according to the present invention, such that the 
camera, illumination device and the surface are in and/or are brought into at least one 
defined geometric relationship with each other during the inspection of each area to be 
inspected on the surface, at least for the period of time required to take a picture. As a 

25 result, a consistent level of picture-taking quality can be attained for every area to be 
inspected, and a high defect detection rate can be attained. 

Preferably, at least one camera and at least one illumination device are located in a 
single inspection unit, so that the geometric relationship between the camera and the 
illumination device in this inspection unit is always specified in a fixed manner. 
30 Particularly advantageously, several cameras can also be provided per inspection unit. 
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the optical axes of which are possibly oriented differently, so that pictures of an area to 
be inspected can be taken sinnultaneously from different perspectives. 

Furthermore, according to the present invention, several cameras and illumination 
devices or inspection devices can each represent separate subsystems, each of which 
5 is connected with the evaluation device and the control device. By inspecting various 
surface areas simultaneously, the inspection time can be reduced considerably. The 
subsystems can also be suitable, in particular, for different-sized inspection areas. 

In a particular embodiment of the inventive system, at least one stationary and one 
movable subsystem are provided. 

10 To identify the specified geometric relationship, it is particularly advantageous when the 
cameras used are calibrated three-dimensionally, so that the relative orientation 
between camera and object can also be determined from the camera image and a 
known position of a known object. The camera is then preferably calibrated with 
reference to illumination devices, the object and displacement devices, so that the 

15 relative positions are known exactly at any time. 

The present invention is explained below in greater detail with reference to an 
exemplary embodiment and the drawing. All of the features described and/or depicted 
graphically are part of the present invention, independently of their wording in the claims 
or their back-references. 

20 Figure 1 is a schematic illustration of an inventive system for inspecting surfaces of 
a three-dimensional body, in a top view; 

Figure 2 shows part of the system in Figure 1 , in a side view, and 

Figure 3 shows typical structures of surface flaws. 

System 1 shown in Figure 1 for inspecting surfaces of a three-dimensional body 2 is 
25 designed to examine a painted surface of bodies. The system is not limited to this 

application, however. Instead, it can be used in general to inspect any types of surfaces. 

The system includes several cameras and illumination devices, which are subdivided 
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into several inspection units 3 and 4 located at separate points. Stationary inspection 
unit 3 is a first subsystem 6 for examining the sides of body 2, and inspection unit 4 is a 
second subsystem 7 for examining the rest of the surface areas. It is also possible to 
provide further subsystems and to adapt the size of the inspection units to the particular 
5 circumstances. Subsystems 6, 7 are located one behind the other along a displacement 
device 9 - designed as a conveyor belt - for body 2, so that body 2 is moved relative to 
stationary inspection unit 3 and, e.g., small-surface area or large-surface area 
inspection unit 4. In addition, inspection unit 4 is mounted on a displacement device 10 
assigned to inspection unit 4, which allows inspection unit 4 to attain any possible 
10 orientation in space. Displacement device 10 is designed as a manipulator or handling 
device that allows inspection unit 4 to be moved in several degrees of freedom around 
various axes of rotation. 

By way of body 2 moved on conveyor belt 9 and via displacement device 10 with 
inspection unit 4, a relative motion between the camera and the illumination device of 
15 inspection units 3, 4 and body 2 is produced, whereby pictures are taken of the areas to 
be inspected on the surface of three-dimensional body 2 at various points in time using 
the cameras of inspection units 3, 4. The pictures that are taken are analyzed in 
evaluation device 1 1 with the aid of image-evaluation algorithms. 

To coordinate the relative motion between three-dimensional object 2 and the cameras 
20 and illumination devices of inspection units 3, 4, a control device 12 is provided that is 
set up such that the camera, the illumination device and the surface are brought into a 
defined geometric relationship with each other during the inspection of each area to be 
inspected on the surface of body 2, at least for the period of time required to take a 
picture. To this end, control device 12 knows, e.g., by performing a measurement with a 
25 sensor 8, the positions of body 2 moving on conveyor belt 9, and inspection units 3, 4. 
Inspection unit 4 mounted on displacement device 10 can also be brought into a 
specified position by control device 12 relative to the position of body 2, in which the 
defined geometric relationship between the surface of body 2 and the camera and 
illumination device of inspection unit 4 is attained. When this relationship is attained, a 
30 picture is taken of the area to be inspected, and the picture is evaluated by evaluation 
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device 1 1 . 

Evaluation device 11 and control device 12 can be realized as separate computers or 
combined into one computer system. 

Figure 2 shows - in a side view from the front - an inspection unit 4 in second 
5 subsystem 7 of entire system 1 for inspecting surfaces mounted on a displacement 
device 10, in particular a manipulator or handling device. Inspection unit 4 can be 
positioned freely in space around various axes of rotation of displacement device 10, so 
that a specified distance and a specified angular relationship to the surface normal of 
the area to be inspected can be adjusted for a certain surface area on body 2. The 
10 defined geometric relationship is then attained. 

Basically, the examination of the surface can also be carried out using only one 
inspection unit that is freely movable in space. The subdivision into several subsystems 
6, 7 located on a conveyor belt 9 of body 2 to be inspected makes time-optimized work 
possible, however, since inspection units 3, 4 can operate at least partially 
15 simultaneously in order to simultaneously scan various areas to be inspected on body 2. 
The relative motion between body 2 and inspection units 3, 4 is coordinated by control 
device 12, so that moving object 2 and all displacement devices 9, 10 are synchronized. 
Subsystems 6, 7 are not limited to two, and each subsystem 6, 7 can have several 
different inspection units. 

20 Instead of inspection units 3, 4, in which both a camera and an illumination device are 
located, it is also possible to locate individual illumination devices and individual 
cameras in a mobile or stationary manner around movable body such that the camera, 
illumination device and the surface are in a defined geometric relationship with each 
other during the inspection of each area to be inspected on the surface, for the period of 

25 time required to take at least one picture. In this case, the task of control device 12 is 
simplified, but the flexibility of system 1 decreases. 

An inspection unit 3, 4 is composed, e.g., of parallel tubes for illumination, with cameras 
provided between them, with which the area illuminated by the tubes can be monitored 
in bright and dark fields. All other combinations of suitable picture-taking devices - in 
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the form of cameras - and suitable illumination devices are also possible. 

The method for inspecting surfaces of three-dimensional body 2 is described again in 
detail below. 

While three-dimensional body 2 moves on conveyor belt 9, various areas to be 
5 inspected are covered by the cameras and illumination devices of inspection units 3, 4. 
Inspection unit 4 can also be brought into a desired position. Control unit 12 recognizes 
the position of body 2 and each of the inspections units 3, 4 in space. Based on these 
known positions, control device 12 determines whether the specified geometric 
relationship between the camera, the illumination device and the area to be inspected 

10 on the surface has been attained. In this case, control device 12 issues a control 

command to take a picture, and the picture is then analyzed in evaluation unit 1 1 with 
the aid of image-evaluation algorithms. The following can be taken into account: The 
contrast, size, the relationship between size and contrast, the geometry and contour, 
and the arrangement in the bright and dark field when bright/dark field illumination is 

15 used. As a result, it is possible to detect main topological surface defects 13 in the paint 
surface on body 2. Topological defects 13 of this type are positive and/or negative 
deformations on the target surface, as shown in Figure 3 as an example. Surface 
defects 13 can be caused by soiling, craters, solvent boils, pin-holes, wet impressions, 
scratches or the like, or by wetting problems in the painting process. 

20 The data detected in the image evaluation are used to classify defects 13. Based on the 
classification of the various defects, it can be determined, among other things, whether 
it is a relevant defect 13 or whether these defects 13 can be disregarded. For instance, 
surfaces that are slightly uneven - similar to the surface of an orange peel - can usually 
be disregarded. A minimum deviation in the surface normal in the area of the defect 

25 from the surroundings around the defect can serve as the criterium therefor. The 

classification can be designed to be adjustable, parameterizable and/or self-learning, in 
particular. In addition, the classification can also distinguish between various defects 
and defect types. The classification is therefore not limited to differentiation between 
defects and non-defects. 
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With the inventive method and corresponding system 1, which is used, in particular, to 
carry out the method for inspecting surfaces of a three-dimensional body, it is possible, 
due to the unambiguously specified, defined geometric relationship between the surface 
of body 2 and the camera and the illumination device assigned to this camera, to always 
5 create optimum conditions for detecting a defect on the surface. 
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Reference numerals: 

1 Systenn 

2 Three-dimensional body, body 

3 Inspection unit 
5 4 Inspection unit 

6 First subsystem 

7 Second subsystem 

8 Sensor 

9 Displacement device, conveyor belt 

10 10 Displacement device, manipulator or handling device 

1 1 Evaluation device, computer 

12 Control device, computer 

13 Surface defect 



